Geometric k-nearest neighbor estimation of entropy and mutual information
نویسندگان
چکیده
منابع مشابه
Geometric k-nearest neighbor estimation of entropy and mutual information
Nonparametric estimation of mutual information is used in a wide range of scientific problems to quantify dependence between variables. The k-nearest neighbor (knn) methods are consistent, and therefore expected to work well for a large sample size. These methods use geometrically regular local volume elements. This practice allows maximum localization of the volume elements, but can also induc...
متن کاملEstimation of Renyi Entropy and Mutual Information Based on Generalized Nearest-Neighbor Graphs
We present simple and computationally efficient nonparametric estimators of Rényi entropy and mutual information based on an i.i.d. sample drawn from an unknown, absolutely continuous distribution over R. The estimators are calculated as the sum of p-th powers of the Euclidean lengths of the edges of the ‘generalized nearest-neighbor’ graph of the sample and the empirical copula of the sample r...
متن کاملk-Nearest Neighbor Based Consistent Entropy Estimation for Hyperspherical Distributions
A consistent entropy estimator for hyperspherical data is proposed based on the k-nearest neighbor (knn) approach. The asymptotic unbiasedness and consistency of the estimator are proved. Moreover, cross entropy and Kullback-Leibler (KL) divergence estimators are also discussed. Simulation studies are conducted to assess the performance of the estimators for models including uniform and von Mis...
متن کاملMutual Information Estimation in Higher Dimensions: A Speed-Up of a k -Nearest Neighbor Based Estimator
We focus on the recently introduced nearest neighbor based entropy estimator from Kraskov, Stögbauer and Grassberger (KSG) [10], the nearest neighbor search of which is performed by the so called box assisted algorithm [7]. We compare the performance of KSG with respect to three spatial indexing methods: box-assisted, k-D trie and projection method, on a problem of mutual information estimation...
متن کاملBayesian Kernel and Mutual $k$-Nearest Neighbor Regression
We propose Bayesian extensions of two nonparametric regression methods which are kernel and mutual k-nearest neighbor regression methods. Derived based on Gaussian process models for regression, the extensions provide distributions for target value estimates and the framework to select the hyperparameters. It is shown that both the proposed methods asymptotically converge to kernel and mutual k...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: Chaos: An Interdisciplinary Journal of Nonlinear Science
سال: 2018
ISSN: 1054-1500,1089-7682
DOI: 10.1063/1.5011683